The present invention relates in general to neural networks, and more particularly, to a random access memory (RAM) circuit for emulating the learning and recognition behavior of neurons.
A biological neuron is a single nervous cell responsive to stimuli through weighted inputs known as synapses. The weighted stimuli are summed and processed through a particular non-linearity associated with the neuron for providing an output signal. The output of the neuron may then be connected to the synapse of the next level neuron forming an interconnection of neurons known as a neural network, the latter of which possesses certain desirable properties including the ability to learn and recognize information patterns in a parallel manner.
Technologists have long studied the advantageous nature of the biological neuron in an attempt to emulate its behavior in electronic circuitry. As indicated, some of the advantages of the neural network include the learning and recognition of patterns and shapes. The neural network may be taught a particular pattern and later be called upon to identify the pattern from a distorted facsimile of the same pattern. Unlike conventional signal processing techniques wherein the solution is programmed with a predetermined algorithm, the recognition technique of the neural network may be learned through an iterative process of adding random noise to the input signal of an ideal pattern, comparing the output signal of the neural network with the ideal pattern, and adjusting the synaptic weights to provide the correct response.
Another desirable feature of the neural network is its robust quality to a localized failure wherein the ability to recognize is simply degraded, typically in proportion to the extent of the failure. This feature is advantageous over conventional signal processing circuits wherein a single component failure may cause a complete system failure. Indeed, the neural network may include a feedback mechanism to alter the synaptic weights in undamaged paths to compensate for the damage and restore the operation of the system to the original level of performance. Furthermore, the neural network inherently processes information in parallel offering real-time execution without the classic von Neumann bottleneck associated with conventional serial processing techniques, i.e., digital signal processing and general microprocessor based applications. In general, neural networks are well suited for applications wherein the solution algorithm is not known or difficult to implement, or where conventional processing techniques are too slow. Notably, neural networks typically are not high precision devices, thus, a certain degree of error is generally present in the output signal.
Modern electronic circuits have achieved some degree of success in emulating the biological neuron. Previous examples of electronic neural networks comprise resistor arrays, floating gates and adaptive logic each of which possess one or more limitation. The resistive array is small and integrable into an integrated circuit (IC), but suffer from non-programmability prohibiting post learning as the system changes. The floating gates and adaptive logic are programmable, but typically offer low density limiting the number of neurons available for use.
Hence, there is a need for a circuit having neuron-like behavior in a high speed, high density package and combining the properties of learning and recognition with programmability whereby the knowledge base of the neural network may be updated.